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I.  IMTOQDUCTION 


In  this  report  we  will  consider  the  separable  nonlinear  optimization 
problem  ' 


fCx) 

A 

n 

w 

g±00 

A 

n 

Ij-1 

a. 

3 

sbj 

j “ 1.  . 

One  frequently  employed  algorithm  for  approximately  optimizing  (1)  is  the 
"Separable  Programming"  algorithm  of  Miller  [1] . This  method  forms  a piece- 
wise  linear  approximation  to  (1)  using  a fixed  grid  of  points  for  each  x^ 

and  then  locally  optimizes  the  approximate  program  using  linear  programing. 

When  all  of  the  functions  f^  and  g^  are  convex  functions,  then 

the  procedure  can  be  extended  to  optimize  over  a variable  grid  thus  giving 
an  arbitrarily  accurate  approximation  in  the  vicinity  of  the  optimal  solution. 
This  procedure,  which  we  will  call  Grid  Linearization  is  described  in  Wolfe 
£23  and  can  be  viewed  as  an  extension  of  the  Dantzig-Wolfe  Decomposition 
Principle  for  linear  programs  in  that  it  uses  a restricted  master  linear 
program  to  optimize  over  the  current  grid  at  any  stage,  and  subprograms 
involving  the  restricted  master  dual  variables  to  generate  new  grid  points 
which  become  new  columns  of  the  restricted  master  at  the  next  stage.  For 
convex  programs  this  procedure  has  been  proved  to  converge  in  Dantzig  [3]. 
The  procedure  is  attractive  because  the  restricted  master  program  is  a 
Generalized  Upper  Bounded  L,  P.  (and  hence  easy  to  solve),  while  the  non- 
linear subprobleras  are  convex  single  variable  problems  (also  easy  to  solve). 


•t 
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In  this  paper  we  consider  the  extension  of  the  grid  linearization  tech- 
nique to  non-convex  separable  programs.  The  result  is  an  algorithm  which 
combines  the  grid  linearization  procedure  with  a branch  and  bound  structure . 
Restricted  master  linear  programs  are  solved  at  each  stage  of  the  branch  and 
bound  search.  Nonconvex  single  variable  subproblems  generate  bounds  for  the 
branch  and  bound  search  and  also  generate  new  grid  points  for  refinement  of 
the  linear  approximation. 

Section  II  of  this  report  gives  a brief  sketch  of  the  grid  linearization 
process,  primarily  to  introduce  notation.  Section  III  develops  the  algorithm 
for  the  nonconvex  case.  Section  IV  discusses  the  relation  of  this  method 
to  other  algorithms  in  the  literature.  Section  V deals  with  computational 
considerations  for  attaining  efficiency. 


II.  BRIEF  REVIEW  OF  THE  GRID  LINEARIZATION  METHOD  (Convex  Case) 
A.  Fixed  Grid 


We 


consider  problem  NLP  (1)  with  fj  (i=l,  . m; 


J = 1,  ...»  n)  all  convex  functions.  For  each  variable  x^ 0=1,  . ..,  n) 
suppose  we  have  chosen  a (temporarily  fixed)  grid  of  points  x.,,  (k  = 0,  . ...,  n.) 

J jL  o 

with 


a.  = x.  < x.,  < ...  < x.,,  < ...  < x.  = b. 
J Jo  jl  jk  jn^  j 


(2) 


Then  any  x^  £ [aj  , b^  ] can  be  written  as  a convex  combination  of  grid  points 


v nl 

xj = s ki,  v. 


'jk 


(3) 


with 


l = 1 


XJk  ~ 0 


If  the  convex  combination  is  such  that  for  every  J at  most  two  are 

non-zero  and  those  two  have  adjacent  k indices,  then  a piecewise  linear 
approximation  to  fj(Xj)  Is  defined  by 


| xjk  fj(xjk)  ( 

* 

and  similarly  for  the  g^(xj)  , 

Thus  If  we  let  fjk  = f^Cx^)  and  = gjjOt^)  * then  the 

separable  programing  method  defines  a linear  program  whose  decision 

variables  are  the  convex  combination  weights  A^  ; 


?.  min  I I A„f 


jk*jk 


5*  T‘  ■]  l *jk®ijk 


* 0 V, 


Xjk  * 0 VJ  • ^k  * 
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An  optimal  solution  to  the  linear  program  P will,  If  the  above  adjacency 

A 

condition  is  satisfied,  provide  an  optimal  solution  to  the  piecewise  linear 
approximation  to  NLP  (1)  with  solution  values  given  by  (3)  . If  NLP  Is 
convex,  then  any  feasible  solution  gives  a feasible  NLP  solution  and 

it  Is  well  known  that  the  optimization  process  will  automatically  result  in 
an  adjacent  solution.  Hence  P^  efficiently  approximates  the  solution  to 

NLP. 

B.  Grid  Refinement 

When  an  optimal  solution  to  has  been  reached,  a natural  question 

Is  whether  there  are  new  grid  points  in  the  vicinity  of  the  solution  which  will 
Improve  the  piecewise . linear  approximation  and  hence  the  accuracy  of  the  optimal 
solution.  The  grid  linearization  process  generates  new  grid  points  as  follows: 
Suppose  A Is  the  optimal  primal  solution  and  Or,  o)  = (ir.^,  ...»  c^,  ...»  a ) 

the  optimal  dual  solution  to  the  L.  P.  P^  . 

A new  grid  point  can  improve  the  solution  only  if  its  reduced  cost 

in  the  current  optimal  tableau  is  negative.  This  reduced  cost  is 

°J  “ l *i  giJ  (xJ ^ " °J  (8) 

Thus  the  grid  linearization  algorithm  solves  the  convex  single  variable 
subproblems 

min  f.(x.)  - l it.  g. , (x  } 

Wh  J 1 J J 


(9) 


5 


for  each  j =1,  . ..,  n with  solutions  x, 


If  for  any  j we  have 


fJ  (V  (£j}<  °J 


then  x^  = Xj  is  a new  grid  point  value  for  Xj  which  may  improve  the 

solution*  If  (10)  is  violated  for  all  j = 1,  ...,  n,  then  the  current 

solution  for  P^  translates  via  (3)  into  an  optimal  solution  for  NLP. 

In  the  Grid  Linearization  algorithm  this  process  is  applied  iteratively, 
alternately  optimizing  P^  (called  the  restricted  master  problem)  and  using 

the  subproblems  (9)  to  generate  refined  grid  points  and  hence  to  generate 
new  columns  which  are  added  to  the  restricted  master  for  the  next 

iteration.  It  has  been  proved  (see  for  example  Dantzig  13])  that  this 
process  is  infinitely  convergent  for  convex  NLP.  Further  description  of 
the  process  and  proofs  of  its  properties  can  be  found  in  Lasdon  £4]  and 
Dantzig  (3]  . 


III.  AN  ALGORITHM  FOR  THE  N0NC0NVEX  CASE 
A.  Lower  Bound 

For  convenience  we  restate  the  NLP  problem  (l) 


min  f(x)  = l f.  (x.) 

j " J 


S.T.  gjU)  <*  l giJ  Uj)  £ 0 i = l,  ... , m 
J 


■a^  S Xj  s bj  J*l,  ....  n 


In  this  section  we  make  no  assumptions  about  convexity  of  the  problem 
functions.  Let 


C = { x j aj  s.  Xj  £ bj  , Vj } ( 11 ) 


and  Cj  = [a^  , b ] . 


(12) 


Define  the  Lagrangian  function  for  NLP  as 


L(x,tt)  = l Lj(Xj  , tt) 

^ m 

■ ] ‘W  ‘i^  *i  eu  H 1 


(13) 


and  note  that  it  is  addititively  separable  in  the  variables  x^  . 

An  important  lower  bound  is  given  in  the  following  Theorem: 
Theorem  1 Let  x be  any  feasible  solution  for  NLP  (l)  . If 


iksQ,  i = 1,  . . . , m 


Then 


min 

xj“j 


Lj  (Xj  , tt)  s t (xj)  . 


(1M 


The  proof  is  a standard  result  from  nonlinear  duality  theory  and  will  not 
be  repeated  here. 

In  particular.  Theorem  1 shows  that 


min  L(x,  ir)  = ][  min 
xeC  J xJeCJ 


ljuj  • w) 


(15) 
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is  a lover  bound  on  the  (global)  optimal  objective  function  value  for 


It  should  be  emphasized  that  l)  this  theorem  is  true  for  any  NLP  - 
no  assumptions  whatever  are  required  (in  particular  convexity  is  not 
required) 

2)  The  Lagrangian  minimization  in  (15)  must  be  a global  minimization. 
Since  L = £ it  suffices  to  be  able  to  globally  minimize  the  single 

variable  (nonconvex)  functions  .. 

3)  The  bound  is  tight  for  well  behaved  convex  programs  in  the  sense 

that  when  n is  dual  optimal,  (l4)  holds  with  equality  V.  . For  non- 

J 

convex  programs,  however,  it  is  well  known  that  a "duality  gap"  may  occur 
so  that  there  may  be  no  feasible  x and  ir  <:  0 for  which 


min  L(x  , u)  = 'l  f,(x.) 
xeC  J J 


L)  The  Lagrangian  minimization  (lM  is  exactly  the  same  as  the 
Grid  Linearization  subproblem  (9)  , 


B.  Feasibility  and  Optimality  in  NLP 

Suppose  we  choose  a grid  of  points  x^  for  each  Xj  j=l,  ...»  u, 

and  set  up  the  restricted  master  linear  program  as  in  (7)  • If  NLP 

is  a nonconvex  program  we  can  no  longer  guarantee  that  X feasible  for 
implies  x given  by  (3)  is  feasible  for  NLP  , nor  can  we  guarantee 

that  optimization  will  automatically  lead  to  an  adjacent  interpolation. 


Nevertheless  P^  is  a linearization  of  NLP  . In  this  section  we  explore 
the  relation  between  feasibility  and  optimality  for  P^  and  feasibility  and 
optimality  for  NLP. 


Theorem  2 Let  X be  primal  optimal  and  (tt  , a)  be  dual  optimal  for  the 
linear  program  P^  (7)  with  objective  function  value  Z . Let.  x*  have 

components  ' x.  = ) X , x.,  as  in  (3)  . Let  x (globally)  solve 
d ^ <3^  J ^ 


min  L(x  , it)  . 
xeC 


1 t 

t 


If  a)  f(x*)  <:  Z 


b)  gi(x*)  js  0 Vi=  i,  m 


c)  L(x  , tt)  :s  £ a 
j=l  0 


Then  x*  solves  NLP  (globally). 


Proof  at  optimality  for  P^  we  have  equal  primal  L.P  and  dual  L.P. 


objective  function  values, 


I 


y v‘iV = z = Sji  °j 


writing  the  dual  to  shows  that  dual  feasibility  requires 


ir  £ 0 


Thus  by  Theorem  1,  (l6)  gives 


L(x  , ir)  i min  { f(x)  j x feasible  for  NLP  } 


combining  (17),  (19),  (20),  (22)  with  feasibility  of  x*  in  NLP  gives 


Z = l a s:  L(£  , ir)  ;s  min{  f(x)  | x feasible  for  NLP}  s f(x*)  s:  Z (23) 


Thus  all  the  above  quantities  are  equal  and  x solves  NLP  . QED 


Theorem  2 gives  conditions  which  are  sufficient  for  optimality  in 
NLP.  These  conditions  are  not,  however,  necessary.  In  particular  they 
will  fail  to  hold  for  any  nonconvex  NLP  which  has  a duality  gap.  The 
primary  value  of  the  theorem  is  that  it  suggests  an  algorithm  for  getting 
closer  to  a solution.  When  condition  c)  is  not  satisfied,  then  some 
Xj  is  a new  grid  point  which  improves  the  approximation  of  to  NLP, 

while  if  a)  or  b)  is  violated,  then  the  problem  is  nonconvex  in  the 


I 


10 


i ;■ 


vicinity  of  x*  and  we  must  resort  to  branch  and  bound.  These  ideas 
will  be  made  precise  in  section  III-C. 

Theorem  2 required  that  possess  an  optimal  solution,  but  it  is 

also  possible  that  may  be  infeasible.  The  following  2 theorems 

explore  this  situation  and  its  implications  for  the  original  problem  NLP. 

Theorem  3 If  P^  is  infeasible,  then  the  dual  to  P^  is  unbounded. 

Proof  The  duality  theorem  of  Linear  Programming  implies  that  the  dual 

to  is  either  unbounded  or  infeasible,  but  tt  = 0 , a j = mjLn  f (x^) 

is  a feasible  solution  to  the  dual.  Hence  it  is  unbounded.  QED  . 


i . > 


Theorem  h Suppose  P^  is  infeasible  and  (u,  a)  + 0(TT1,a1)  (0  £ 0) 
describes  a dual  feasible  ray  along  which  the  dual  to  P^  becomes 
unbounded.  Suppose  x solves  the  Lagrangian  minimization 


i l 
} 

*'  ) 
{ !■ 


min  f(x)  ff.  g. (x) 
xeC  i . • , 


aad  x1  solves  the  related  minimization 


(24) 


F 


If 


min  - \ tt . 1 g.  (x). 
xeC  i 


f(fi)  “ I TT,  g.(x)  fe  J 0 . 
i 1 1 • J ° 


(25) 


(26) 
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and  - J n. 1 g. . (x1)  i J o,1  (27) 

i J J 

then  13 L?  is  infeasible. 

Proof  3y  (2t),  (25),  (26),  and  (27)  we  have,  for  any  xeC  , and  for 
any  6 2:  0 


f(x)  - I * Q^1)  g.(x)  2:  (28) 

min  { f(x)  - J (ir.  + 071. *)  g.  (x)}  2: 
xeC  i 1 1 1 

{min  f(x)  -sj  tt  g (x)}  + 6 { min  - l it,1  g.  (x)}'^ 
xeC  i 1 xeC  i 1 1 

| (“j  * toji)  • 

Thus  for  any  gridpoints  x^  e which  we  might  choose,  the  resulting 
P,  still  has  (tt  a)  + 8(tt1  , a1)  as  a dual  feasible  ray  along  which  the 

. A 

dual  objective  function  is  unbounded,  and  hence  this  is  infeasible. 

But  if  xeC  is  feasible  for  NLP,  then  choosing  its  components  x^  be 
grid  points  must  give  a feasible  . Hence  NLP  is  also  infeasible.  ^iED. 
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C.  The  Algorithm 

The  algorithm  proposed  in  this  section  is  a branch  and  bound 
method.  Branching  is  done  by  dividing  the  interval  = [a^  , b^j  for 

some  variable  x,  into  two  subintervals.  At  each  stage  t of  the  search 
J 

a linearized  problem  over  some  subintervals  is  solved. 

Lagrangian  minimizations  (l4)  then  provide  l)  an  optimality  test, 

2)  an  infeasibility  test,  3)  (perhaps)  . new  grid  points  for  incorporation 
into  P^  as  well  as  4)  a new  lower  bound  on  the  optimal  value  of  NLP 

t 


restricted  to  Xj  e Cj  . 


The  detailed  description  of  the  algorithm  follows: 


Step  1 Initialization 

For  each  j = 1,  ...»  n choose  an  initial  grid  as  Xjo  » a^  , 

= bj  . Let  P^  with  t = l(=  oubproblem  counter)  be  the  P^  program 

corresponding  to  this  initial  grid.  Let  Cj  = , b^J  . Let  L = - 00 

be  the  current  largest  lower  bound  for  . Let  F°  = +°°  be  the  value 

of  f(x)  for  the  best  incumbent  feasible  solution  to  NLP  found  so  far. 

Place  P^1  on  a list  of  subproblems  and  go  to  step  2. 


Step  2 Linear  Program 

If  the  list  of  aubproblems  is  empty,  stop.  The  incumbent  solution  is 
global  optimal.  Otherwise  select  from  the  list  of  subproblemo  the  problem 
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t t 

P^  with  the  smallest  lower  bound  L . 


t 


Solve  this  linear  program  yielding  optimal  value  Z with 


optimal  primal  variables  X and  optimal  dual  variables  ir  , a 


t . 


[If  is  infeasible  the  solution  yields  a dual  feasible  ray 


(it  , a)  + 0(ti  , a1) 


along  which  the  dual  is  unbounded] 


Go  to  step  3. 


Step  3 Lagrangian  Minimization 

Solve  the  n single  variable  problems  min  , L,  (x.  , tt)  giving 

Vcj  0 


solutions  x. 


[If  P,  was  infeasible  also  solve  min  , 


- 1 g^Uj)  giving 


solutions  xj  . ] 


Compute  3 


• | Lj  (lj  • ' > 


I 


If  B ;s  F°  then  immediately  fathom  P^t  and  go  to  step  2. 

If  P > B > L then  increase  the  value  of  the  bound  for  p*  to 

A 

L1  = B and  go  to  step  k . Otherwise  go  to  step  4 without  changing  the  bound. 
Step  ^ Mew  Grid  Points 

For  each  j - 1,  ...,  n,  if  Lj(x.j  , tt)  < a ^ then  incorporate  5^  as  a 
new  gridpoint  for  Xj  in  the  subproblem  P^  . Place  the  new  p * sub- 
problem on  the  list  and  go  to  step  2.  Otherwise  go  to  step  5. 

[For  an  infeasible  P^  , if  all  , u)  is  and  if 

“ ^i1'6 ij(*V  < aj1  then  incorporate  x^1  as  a new  grid  point  for  p * 
and  go  to  Step  2. 

If  all  - Jtk1  g^fxj1)  is  Oj1  also,  then  fathom  P^  since  (by  theorem  J*) 


the  corresponding  NLP  subproblem  is  infeasible.] 


Step  5 Optimality  Test 


Compute  x*  from  A using  (3)  . If  g^x*)  s 0 , i = 1 * , and 


f(x*)<  F then  replace  F°  with  f(x#)  and  let  x*  be  the  new  incumbent 
solution. 

If  a. ) f(x*)  s:  Z 

and  b.)  gi(x*)  SO  i = 1 m,  then  (by  Theorem  2)  x*  is  global 

optimal  for  the  NLP  subproblem  over  xeC^  , Go  to  step  2. 

If  a)  or  b)  is  violated  go  to  step  6. 


Step  6 Branch 

Let  I = { i | g ^£x#)  > 0 Let 

♦j = h*  fj  k “4  *ij = hi  e«k 


Let  l be  the  subscript  j =1  n which  solves 


...  m“  ntfjuj*)  ""j ; su  h‘y-*u 1 

J— X • • %U 

iel 


Let  x*  be  a new  grid  point  for  x^  and  define  two  new  subproblems. 


a. ) 


"t  # 

restricted  to  x^  £ x^ 


/include  only  grid  points 
^to  the  left  of  x£) 


i 

1 


. A 

il 

■i 

t 

•V 


b.) 


restricted  to 


xl*  *1 


/include  only  grid  points 
^to  the  right  of  x*) 


% 


Let  the  bound  for  each  of  these  problems  be  L and  place  both  on  the  list. 
Go  to  step  2 . 


IV.  RELATION  TO  OTHER  METHODS 

The  algorithm  proposed  in  Section  III  is  related  to  several  other 


! 

i 


4 

4 

It 

. .* 


computational  methods  for  separable  nonlinear  optimization.  One  set  of 
relationships  can  be  seen  by  considering  other  algorithms  whose  fundamental 
mechanism  is  the  grid*»lineari  nation  representation. 
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For  convex  programs  representation  with  fixed  grid  is  one  of  the 

oldest  and  most  used  nonlinear  programming  techniques  £lj  . The  general- 
ization to  a variable  grid  for  convex  programs  £2]  is  the  nonlinear  analogue 
for  the  Dantzig  Wolfe  decomposition  principle  £5]  . 

Nonconvex  programs  with  a fixed  grid  were  considered  by  Falk  £6]  and 
Beale  and  Tomlin  £7]  where  branch  and  bound  was  used  to  force  adjacent 
interpolations.  The  current  method  is  the  natural  culmination  of  a 
variable  grid  and  nonconvex  problems. 

Another  set  of  relationships  is  with  other  existing  branch  and  bound 
methods  for  nonconvex  optimization.  A significant  contribution  here  was 
the  work  of  Falk  and  Soland  £8J  and  Soland  £93  who  used  convex  envelopes 
of  nonconvex  functions  to  form  a convex  approximating  problem  which  was 
then  imbedded  in  a branch  and  bound  structure.  Our  method  is  similar  except 
that  the  convex  envelope  problem  is  replaced  by  a sequence  of  improving 

linear  approximations.  The  P^  problems  are  easier  to  formulate  and 

to  solve,  but  they  lack  the  property  of  being  a consistent  underestimate  of 
the  original  problem  functions.  As  a result,  the  bounds  for  our  problem 
are  derived  from  Lagrangian  duality  in  contrast  to  the  Falk  and  Soland 
bounds  which  derive  directly  from  the  convex  envelopes.  Greenberg  £1CQ 
indicates  that  Lagrangian  bounds  are  stronger  than  convex  envelope  bounds 
in  some  circumstances.  If  all  the  problem  functions  are  concave,  then 
convex  envelopes  are  the  same  as  linear’  interpolations  between  the  end- 
points  of  the  intervals  C.  . In  this  case  our  algorithm  is  very  similar 

to  that  of  Soland  £9]  , and  step  ^ would  never  occur.  Another  similar 
branch  and  bound  method  for  the  concave  case  with  linear  constraints  is 
due  to  Walkup  [11]  . 


IT 


V COMPUTATIONAL  CONSIDERATIONS 

Implementation  of  the  algorithm  outlined  in  section  III  involves 
three  distinct  computational  requirements. 


a. ) Solve  the  linear  programs 


b.)  Solve  the  single  variable  nonconvex  Lagrangian  minimizations 


c.)  Generate  and  maintain  the  problem  list  required  by  the 
Branch  and  Bound  structure. 

In  this  section  we  discuss  each  of  these  briefly  indicating  possible 
choices  and  tradeoffs  which  might  influence  the  efficiency  of  the  procedure. 

The  linear  programs  P- ^ which  must  be  solved  in  step  2 of  the 

A 

algorithm  have  m + n constraints  and  as  many  variables  as  there  are  grid 

"t  r k 

points  in  the  subrectangle  C . The  n convexity  constraints  = 1 , 

k J 

tyj  can  be  handled  implicitly  by  a Generalized  Upper  Bounding  algorithm, 

so  the  effective  basis  size  is  only  m . Any  sparsity  in  the  original  NLP 
constraints  (g^(xj)  = 0)  is  inherited  in  the  first  m constraints  of  P^  . 

Thus  P-j^  is  a linear  program  which  may  have  substantial  structure  and  which 

should  not  be  too  difficult  to  solve.  When  new  grid  points  are  added  to  an 
existing  the  existing  solution  provides  a natural  advanced  start  for 

the  new  optimization.  Another  possibility,  when  the  number  of  grid  points 
becomes  excessive  is  to  drop  non-basic  grid  point  columns  from  the  problem. 
However,  for  the  convex  case  this  destroys  the  convergence  proof. 

In  Step  3 of  the  algorithm  we  must  perform  the  single  variable  Lagrangian 

minimization  of  L^Uj  , w)  * f^(xj)  - g^  (x^)  over  the  interval 


Xj  e Cj  . If  fj  and  the  g^j  are  all  convex  (and  tk  £ 0)  then 

is  convex  in  x^  and  the  minimization  can  be  easily  handled  by  methods 

such  as  Fibonacci  search  or  perhaps  even  analytically  by  setting  the 
derivative  to  zero.  If  fj  and  the  g^  are  concave,  then  i3  concave 

also  and  one  endpoint  of  will  be  minimal.  In  the  general  case  where 

Lj  is  neither  convex  nor  concave,  the  problem  of  globally  minimizing  Lj 

over  an  interval  is  not  trivial.  Most  of  the  existing  methods  are  heuristic 
in  nature,  but  if  bounds  on  the  derivatives  of  are  known,  then  a 


minimax  optimal  search  plan  due  to  Shubert  {12]  can  be  used.  In  any  case 
these  are  single  variable  minimizations  over  an  interval  and  should  be 
substantially  easier  than  a direct  n-dimensional  search  for  the  solution 
to  NLP. 

In  most  branch  and  bound  algorithms  there  are  tradeoffs  between  solution 
strategy  and  required  storage,  and  these  tradeoffs  affect  the  efficiency  of 
the  resulting  algorithm.  This  algorithm  is  no  exception.  There  are  two 
principal  tradeoffs  to  be  considered.  The  first  is  related  to  which  sub- 
problem on  the  list  should  be  solved  at  any  given  iteration.  As  the 
algorithm  is  written,  the  "most  promising"  subproblem  (smallest  L^}  is 
attacked  at  each  iteration.  It  is  easy  to  imagine  situations  in  which 
two  distinct  equal  valued  global  minima  exist  and  the  algorithm  would 
spend  much  time  switching  back  and  forth  between  the  respective  sub- 
problems doing  very  little  work  on  each  at  a given  iteration.  It  might 
be  better  to  do  more  work  on  a given  subproblem  to  avoid  so  many  switches 
even  if  this  means  temporarily  working  on  a subproblem  which  is  not  the 
most  promising. 
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The  answer  to  this  question  depends  on  how  much  trouble  it  is  to.  set 
up  a new  subproblem.  This  depends  on  how  much  information  is  stored  about 
the  subproblem  and  previous  solutions  to  it.  Some  choices,  in  decreasing 
order  of  storage  requirement  are  the  entire  tableau,  the  previous  optimal 
basis  inverse,  the  previous  optimal  basis  vectors,  or  just  the  grid  points 
which  define  the  problem.  There  is  clearly  a tradeoff  here  between  storage 
space  and  solution  speed.  The  decision  which  is  made  in  any  particular 
case  must  depend  on  the  computational  facilities  available  and  experience 
with  the  class  of  problems  to  be  solved. 

VI.  AREAS  FOR  FURTHER  STUDY 

This  report  has  presented  the  outline  for  an  algorithm  which  solves 

* 

separable  nonconvex  optimization  problems  using  linear  subproblems.  The 
method  has  close  relationships  to  several  existing  optimization  methods, 
but  also  some  desirable  advantages  over  them.  There  are  several  areas 
which  require  further  investigation: 

a.  ) At  the  moment  the  convergence  properties  of  the  method 

are  unknown.  An  effort  to  resolve  the  question  is 
currently  underway, 

b.  ) Since  the  restricted  master  problems  are  linear,  it 

should  be  possible  to  exploit  special  structure  in  NLP  to 
a considerable  degree.  We  plan  to  investigate  this  in 
the  near  future. 

c. )  Computational  behavior  of  the  method  on  particular  classes 

of  nonconvex  problems  is  of  interest. 

d.  ) As  computational  experience  accumulates  the  questions  of 

branch  and  bound  organization  raised  in  section  V should 
be  resolved. 
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e.)  For  fixed  grid  problems  Beale  and  Tomlin  I/?J  have  shown  how 
strong  bounds  can  be  derived  directly  from  the  optimal 

tableau.  Possible  extension  to  the  variable  grid  case  should 
be  investigated. 
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